Refactor IssueHandler from supervisor to issue_verification_agent #473
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces the Issue Verification Agent, migrating the post-fix lifecycle management from the supervisor to a dedicated BeeAI workflow. Key additions include modules for baseline test reproduction, XUnit comparison, and a testing analyst agent, along with several new tools for interacting with Jira, GitLab, the Errata Tool, and Testing Farm. Feedback highlights a critical bug in the Jira attachment tool call involving incorrect data structures and encoding. Performance improvements were also requested to wrap blocking synchronous calls in threads within asynchronous methods, and a simplification was suggested for unreachable code in the GitLab MR search logic.
TomasTomecek
left a comment
TomasTomecek
left a comment
There was a problem hiding this comment.
wow, this is massive, thank you for working on this
are you also planning to drop the former supervisor code?
btw this definitely needs rebase against upstream/main
|
After the errata agent is moved out of supervisor, the supervisor will be dropped altogether of course. I reckon it's on the safe side to keep the code in there for the moment. Yeah, some stuff landed since I started working on it so rebase it needs. |
martinky82
force-pushed
the
main
branch
2 times, most recently
from
024e20f to
03eab1a
Compare
|
I rebased the PR and now I have the pre-commit installed as well, so it should be in better shape than the first attempt. |
TomasTomecek
left a comment
TomasTomecek
left a comment
There was a problem hiding this comment.
please address the bug with the issue_key, the other 2 comments are mostly cosmetic
| return JSONToolOutput(result=results) | ||
|
|
||
| except Exception as e: | ||
| from beeai_framework.tools import ToolError |
There was a problem hiding this comment.
Could you, please, elaborate?
|
Additional note, please implement makefile target for this so it can be triggered as the other agents. For example like we do with triage |
TomasTomecek
left a comment
TomasTomecek
left a comment
There was a problem hiding this comment.
I used opus to review this PR and overall it's very well done there is just one internal API breakage:
Broken caller of get_jira_dev_status in zstream_search.py
The return type of GetJiraDevStatusTool and GetJiraPullRequestsTool changed from a bare list to a dict wrapper ({"commits": [...]} / {"pull_requests": [...]}). preliminary_testing_agent.py was updated to handle
this, but zstream_search.py was missed.
In ymir/tools/privileged/zstream_search.py around line 235-244:
dev_status = await run_tool(GetJiraDevStatusTool(), issue_key=issue_key)
commits = json.loads(dev_status) if isinstance(dev_status, str) else dev_status
# ...
commit_urls = [_get_patch_url(c["url"]) for c in commits if c.get("url")]After this change, commits will be {"commits": [...]} instead of a list, so iterating it will iterate over dict keys ("commits") rather than commit objects, causing a KeyError.
The test mocks in ymir/agents/tests/unit/test_zstream_search.py also still return bare lists, so the tests pass but don't catch this.
Otherwise LGTM, thank you for working on this!! Looking fwd to trying this out.
Good catch, thanks! Fixed in the last commit. |
Makefile target is implemented, thanks for the hint. |
TomasTomecek
left a comment
TomasTomecek
left a comment
There was a problem hiding this comment.
LGTM, though this needs a rebase
Rebased |
Move the IssueHandler workflow from ymir/supervisor/ into a standalone BeeAI Framework agent at ymir/agents/issue_verification_agent.py, following the pattern established by preliminary_testing_agent. This includes: - New agent: issue_verification_agent with Workflow-based state machine - Testing analyst inlined as a sub-agent for AI-driven test result analysis, using ReasoningAgent with maintainer rules from MCP - New MCP tools: errata (GetErratum, GetErratumBuildNvr), testing_farm (GetTestingFarmRequest, ReproduceTestingFarmRequest), gitlab (SearchGitlabProjectMrs), jira (UpdateJiraComment, AddJiraAttachments, GetJiraAttachment) - New unprivileged tools: read_logfile, read_readme, search_resultsdb, analyze_ewa_testrun - Supporting modules moved to ymir/agents/utilities/ - Shared types added to ymir/common/models.py (JotnarTag renamed to YmirTag) - Skill definition for issue_verification with full workflow details - Makefile target and compose service for standalone execution - Jira dev-status tools return dict-wrapped responses for MCP compliance - Blocking sync calls wrapped in asyncio.to_thread - URL allowlisting removed from read_logfile (egress control handles security) The supervisor's ErratumHandler remains unchanged for a follow-up. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TomasKorbar
force-pushed
the
resolve-conflicts
branch
from
e685cdc to
e9daa3a
Compare
3 participants
TODO:
packit/packit.dev.Fixes
Related to
Merge before/after
RELEASE NOTES BEGIN
Packit now supports automatic ordering of ☕ after all checks pass.
RELEASE NOTES END